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ABSTRACT 

The  use  of  near,  mid-wave  and  long-wave  infrared  imagery  for  the  detection  of  mines  and  concealed  weapons  is 
demonstrated  using  several  techniques.  The  fusion  algorithms  used  are  wavelet  based  fusion  and  Fuzzy  Logic 
Approach  (FLA)  fusion.  The  FLA  is  presented  as  one  of  several  possible  methods  for  combining  images  from 
different  sensors  for  achieving  an  image  that  displays  more  information  than  either  image  separately.  Metrics  are 
suggested  that  could  rate  the  fidelity  of  the  fused  images,  such  as,  a  textured  clutter  metric  and  entropy  metric. 

1.  INTRODUCTION 

Even  after  the  hostilities  are  over  from  an  armed  conflict,  armed  mines  are  still  left  buried  under  ground.  A  requirement 
exists  within  the  U.S.  Army  to  develop  technologies  to  detect  and/or  clear  mine  fields.  There  are  several  problems  with 
existing  sensor  detection  technologies,  for  example;  there  is  no  ‘silver  bullet’  technology  or  sensor  that  detects  all  types 
of  mines.  Some  sensors  do  not  image  well  through  wet  soil,  others  have  problems  with  a  good  signal  to  noise  or  clutter 
ratio,  the  scanning  time  of  the  sensors  is  usually  slow  and  technologies  are  not  well  suited  for  wide  area  searching. 
Sensor  fusion  offers  a  way  to  improve  the  signal-to-clutter  ratio  by  combing  the  images  from  several  wave  bands. 

The  same  algorithms  that  are  used  for  fusion  of  multiband  imagery  for  mine  detection  can  also  be  used  for  the  purpose 
of  concealed  weapon  detection.  As  is  the  case  of  mine  detection,  concealed  weapon  detection  ideally  should  be  reliable 
and  fast,  however,  as  with  any  imaging  and  detection  technology,  there  are  tradeoffs  to  consider  that  involve  technical 
as  well  as  social  issues.  Portability  and  imaging  speed  place  constraints  on  the  size  and  resolution  of  sensors  and  the 
computer  systems  that  are  used  to  control  the  imaging  devices  for  both  mine  detection  and  concealed  weapon  detection. 

The  authors  have  obtained  infrared  images  of  buried  mines  from  sensors  in  several  wavelength  regimes  and  describe  in 
this  paper  the  fusion  of  multiband  infrared  images.  Combined  with  the  aforementioned  images,  the  authors  have  tested 
several  computational  methods  to  combine  and  enhance  the  images  from  the  sensors,  such  as  wavelet  edge  processing, 
fuzzy  logic  fusion,  Gaussian  Laplacian  pyramid  fusion,  and  others.  The  advantages  of  using  the  infrared  band  for  mine 
detection  are  the  following:  infrared  can  detect  both  surface  and  buried  mines,  images  can  be  obtained  through  wet  soil, 
there  is  shape-based  discrimination  possible  and  that  can  be  enhanced  with  wavelets,  and  can  be  obtained  at  night. 
Disadvantages  are  mines  buried  for  a  long  time  loose  thermal  contrast,  damp  weather  can  also  reduce  contrast  and  there 
is  the  problem  of  clutter  due  to  debris  in  the  imaging  area.12 
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2.  METHOD 


The  images  were  combined  by  using  either  the  MATLAB  Fuzzy  Inference  System  (FIS)  combined  with  the  fuzzy  logic 
approach  (FLA)  or  the  FuseTool  interface.  In  his  book  Multi-Sensor  Fusion  ',  Brooks  points  out  that  fuzzy  logic  is  a 
technology  that  shows  “promise  for  use  with  sensor  problems.”  He  goes  on  to  mention,  however,  that  because  of  the 
numerous  forms  of  membership  functions,  methods  of  recombination,  etc.,  it  is  difficult  to  know  exactly  which 
implementation  is  best  suited  for  use  in  sensor  fusion  technology.  The  fuzzy  logic  approach  to  an  image  fusion 
algorithm  is  described  as  one  option  for  fusion  below.  Future  papers  will  define  and  compare  other  methods  and 
algorithms  for  image  fusion. 

A  great  deal  of  interest  has  been  shown  in  the  Fuzzy  Logic  Approach  (FLA)  during  the  last  three  decades  for  use  in 
numerous  technical  areas  2’ 3'  4' 5'  6.  A  strong  point  of  the  FLA  is  that  it  permits  the  encoding  of  expert  knowledge 
directly  and  easily  using  rules  with  linguistic  labels.  A  weak  point  is  that  it  usually  takes  some  time  to  design  and  tune 
the  membership  functions  that  quantitatively  define  the  parameters  of  interest.  To  enable  a  system  or  process  to  deal 
with  system  level  uncertainties,  researchers  have  incorporated  the  concept  of  fuzzy  logic  into  many  control  systems.  It 
has  been  found  that  artificial  neural  network  learning  techniques  can  automate  this  process  and  substantially  reduce 
development  time  while  improving  performance  5 .  In  this  paper  the  authors  demonstrate  sensor  fusion  for  the  purpose 
of  mine  detection  and  weapon  detection  using  the  FLA  and  Gaussian  Laplacian  wavelet  fusion. 

The  basic  algorithm  for  pixel  level  image  fusion  using  the  fuzzy  logic  approach  is; 

•  Read  first  image  in  variable  il  and  find  its  size  (rows:  zl,  columns:  si). 

•  Read  second  image  in  variable  i2  and  find  its  size  (rows:  z2,  columns:  s2). 

•  Variables  il  and  i2  are  images  in  matrix  form  where  each  pixel  value  is  in  the  range  from  0-255.  Use  gray 
color  map. 

•  Compare  rows  and  columns  of  both  input  images,  starting  from  the  upper  left.  If  the  two  images  are  not  of  the 
same  size,  select  the  portions  which  are  of  same  size. 

•  Convert  the  images  in  column  form  which  has  C=  zl*sl  entries. 

•  Make  a  Fuzzy  Inference  System  file  which  has  two  input  images,  (See  Fig.’s  1  and  2). 

•  Decide  the  number  and  type  of  membership  functions  for  both  the  input  images  by  adjusting  the  membership 
functions.  Input  images  in  antecedent  are  resolved  to  a  degree  of  membership  between  0  to  255. 

•  Make  rules  for  two  input  images  which  resolves  the  two  antecedents  to  a  single  number  from  0  to  255. 

•  For  num=l  to  C  in  steps  of  one,  apply  fuzzification  using  the  rules  developed  above  on  the  corresponding  pixel 
values  of  the  input  images  which  gives  a  fuzzy  set  represented  by  a  membership  function  and  results  in  output 
image  in  column  format. 

•  Convert  the  column  form  to  matrix  form  and  display  the  fused  image. 


Fig.  1  Mamdani  FLA  FIS 


Fig.  2:  Mamdani  MF’s 


Fig.’s  1  through  4  show  the  various  MATLAB  interfaces  that  are  generated  as  part  of  the  FLA.  Fig.  1  is  the  interface 
picture  showing  that  the  Mamdani  approach  with  two  input  variables  representing  two  wavebands  is  to  be  used.  Fig.  2 
indicates  the  use  of  Gaussian  Membership  functions  to  model  the  interaction  of  the  two  waveband  inputs.  Fig.  3  below 
shows  the  firing  of  the  membership  functions  over  the  prescribed  input  wavelength  ranges  and  Fig,  4  below  shows  the 
resulting  3D  image. 


Fig.  3:  FIS  firing  diagram 


Fig.  4:  FIS  output  surface 


Wavelet  Enhancement  of  Fused  Images 


A  fairly  old  method  of  computing  a  local  spectrum  is  to  apply  the  Fourier  Transform  (FT)  to  one  specific  piece  of  the 
signal  at  a  time.  This  is  the  idea  behind  what  is  called  the  Windowed  Fourier  Transform  (WFT).  Basically,  the 
implementation  involves  using  a  rectangular  window  to  isolate  a  portion  of  the  signal  of  interest,  which  is  then  Fourier 
transformed.  As  the  window  slides  along  to  different  positions,  the  WFT  gives  the  spectra  at  these  positions.  This  kind 
of  analysis  has  a  fundamental  problem  however,  similar  to  the  Heisenberg  Uncertainty  Principle  in  Quantum  Mechanics 
(QM).  Multiplying  the  signal  by  a  window  function  results  in  convolving  or  mixing  the  signal  spectrum  with  the 
spectrum  of  the  window.  Add  this  to  the  fact  that  as  the  window  gets  smaller,  its  spectrum  gets  wider,  and  we  have  the 
basic  dilemma  of  localized  spectra:  the  better  we  determine  the  position  of  the  signal,  the  poorer  we  localize  the 
spectrum.  This  is  analogous  to  the  case  in  QM  where  increased  precision  in  a  description,  of  say,  the  momentum  of  an 
electron  reduces  the  precision  available  of  the  position  of  that  electron.  Very  accurate  determinations  can  be  made  or 
computed,  but  both  are  not  available  to  an  unlimited  degree  of  precision.  Correspondingly,  there  is  a  fundamental 
physical  limit  to  the  degree  of  precision  of  the  frequency  content  of  a  signal  at  a  particular  position  [7,8]. 

In  1946  Dennis  Gabor  [8]  introduced  a  version  of  the  WFT  that  reduced  this  uncertainty  somewhat.  The  Gabor 
transform  uses  a  Gaussian  profile  for  the  window  since  the  Gaussian  is  the  function  that  minimizes  this  uncertainty. 
However,  the  underlying  idea  of  localizing  a  spectrum  of  a  signal  by  windowing  the  signal  needs  to  be  reconsidered. 
Obviously,  care  must  be  taken  in  the  selection  of  the  signal.  Careful  attention  to  the  placement  of  the  window  however, 
is  not  an  easy  task  for  realistic  time-varying  signals.  We  are  in  fact  trying  to  do  two  different  things  at  once.  Frequency 
is  a  measure  of  cycles  per  unit  time  or  signal  length.  So  that  high  frequency  oscillations  take  much  less  signal  length  or 
time  than  do  low  frequency  oscillations.  High  frequencies  can  be  well  localized  in  the  overall  signal  with  a  short 
window,  but  low  frequency  localization  requires  a  long  window.  The  wavelet  transform  takes  an  approach  that  permits 
the  window  size  to  scale  to  the  particular  frequency  components  being  analyzed. 


The  basic  flow  of  processing  in  wavelet  analysis  is  shown  below  in  Fig.  5: 


Fig.  5:  Wavelet  processing  flow  diagram 

4.  IMAGES 

The  figures  below  are  samples  of  the  raw  input  images  from  the  infrared  sensors.  Fig.  6  is  mid-wavelength  version  and 
Fig.  7  is  the  long-wavelength  image  of  the  mine  field.  Fig.’s  8  and  9  are  the  fused  images  using  the  two  input  images 
and  the  wavelet  enhancement  and  FLA  respectively.  Fig’s  10  and  11  are  long-wave  images  taken  during  the  day  and 
night  and  then  fused  in  Fig.  12. 


Fig.  6:  amber_mwr 


Fig.  7:  thermovision  1000 


Fig.  8:  fused_fusetool_contrast 


Fig.  9:  fused_fuzzy 


Fig.  10  daytime  IR 


Fig.  1 1 :  nighttime  IR 


Fig.  12:  day  and  night  fused 


A  program  was  written  to  take  the  input  of  two  or  more  sensors  and  then  build  a  map  of  the  mine  location  also  using  the 
FLA.  Fig.  11  is  a  ‘dummy’  output  graph  in  the  sense  that  it  was  generated  with  synthetic  sensor  data,  however,  the  idea 
is  evident.  The  purpose  of  the  graph  is  to  demonstrate  the  potential  capability  of  such  a  fuzzy-based  detection  system  to 
decide  and  then  display  where  buried  mines  of  different  types  are  ;  see  Appendix  1  for  the  code. 


Fig.  11:  Output  of  mine  detection  program 


Fig.  12:  visual  image11 


Fig.  13:  passive  mmwave  11  Fig.  14:  FLA  fused 


Fig.’s  12  and  13  show  the  image  of  a  concealed  weapon  with  visual  band  and  passive  millimeter-wave  (mmwave). 11 
Passive  millimeter-wave  forms  an  image  from  the  radiation  emitted  from  the  body  by  virtue  of  it’s  temperature,  no 
additional  incident  radiation  is  needed.  The  image  in  Fig.  14  is  fused  using  a  Fuzzy  Logic  Approach  (FLA)  fusion 
algorithm  to  segment  the  location  of  the  handgun. 


An  entropy  and  texture  based  clutter  metric  were  run  over  the  images  to  see  if  there  was  a  correlation  of  the  visual 
quality  of  the  fused  imagery  with  the  metrics.  The  entropy  of  an  image  is  a  measure  of  the  information  content,  in  terms 
of  gray  scale  levels,  and  is  also  related  to  the  texture  of  the  image.  The  maximum  value  the  entropy  metric  can  take  on 
is  eight  and  the  minimum  is  zero.  The  equation  used  for  the  calculation  of  the  entropy  [10]  is  shown  below,  and  the 
values  for  the  metrics  are  shown  in  Table  I. 


L-l 

H=-  I  p(g)  log2  p(g) 
g=0 

wherep(g)is  the  probability  of  gray  value  g,  and  the  range  of  g  is  [0,...,L-1], 


Table  I:  Image  Metrics 


Filename 

Entropy 

Text  Clut 

a  m  b  e  r_mwr_20_c  ut 

6.0973 

38.96 

daytimeIR 

7.1857 

127.09 

fused  fusetool  maximum 

7.3548 

91.13 

fused  fuzzy  trim 

6.3736 

36.49 

nightimeIR 

7.4852 

80.1 

thermovisionIQOO  1  cut 

6.3121 

36.97 

5.  CONCLUSIONS 

In  summary,  the  authors  have  shown  some  of  the  sensor  combinations  and  algorithms  that  can  be  combined  to  detect 
buried  mines  and  concealed  weapons.  Which  sensors  are  used  is  going  to  be  determined  by  factors  such  as  cost,  false 
alarm  rate,  and  detector  material  costs.  Using  passive  infrared  and/or  millimeter-wave  multiband  imagery  with  sensor 
fusion  and  edge  enhancement,  it  is  possible,  in  some  circumstances,  to  detect  buried  mines  and  weapons  concealed 
under  clothing.  When  imaging  through  thick  layers  and  a  high  degree  of  resolution  is  required,  the  best  sensor  may  be 
active  millimeter-wave  imagers,  however,  this  capability  brings  with  it  greater  cost.  The  authors  have  shown  some  of 
the  imaging  possibilities  using  passive  infrared  imagery  and  millimeter-wave  images.  Future  research  could  show  how 
the  passive  infrared  sensors  used  with  image  fusion  and  processing  algorithms  can  detect  buried  mines  in  cluttered 
fields  and  concealed  weapons  such  as  handguns. 
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Appendix  1 

Mine  Detection  Code 


clear  all 

close  all  %Closes  windows 

mineland  =  imread ( ' amber_mwr_20_cut . jpg ' ) ;  %Loads  jpg  image 

thold=0 ; 
f reql=0 ; 
freq2=0 ; 

%Request  input  values 
while (thold<400) |  ( thold>700 ) 

thold  =  input (' Enter  the  threshold  value  for  the  explosive  (between  400-700) 

1  )  ; 
end; 

while ( freql<500 ) |  (freql>999) 

freql  =  input ( 1  Enter  minimum  frequency  for  plastic  mines  in  Hz  (between  500 
999):  '); 

end; 

while ( freq2<500 ) |  (freq2>999) 

freq2  =  input (' Enter  minimum  frequency  for  Metal  mines  in  Hz  (between  500- 
999):  '); 

end; 

fl= freql/ 1000  ; 
f 2  =  freq2/1000  ; 
t=thold/1000 ; 

%Display  image,  labels... 
figure  (1) ; 
imshow (mineland, [] ) 
axis  (  '  of  f 1 ) ; 

title  ('The  land  area  to  be  scanned  for  mines') 


[n,m,k]  =  size (mineland) ;  %Get  no.  columns,  rows,  color  depth 
copy=mineland ( %Copy  contains  image  now 
%///// 

% [qq  qgg] =size (copy) ; 

%FFTcopy=f f t2 (copy) ; 

%MagFFTcopy=abs (FFTcopy) ; 

%AngFFTcopy=angle ( FFTcopy) ; 

% [pi , ql ] =size ( AngFFTcopy ) ; 

%/////// 

F=rand(n,m);  %Generate  random  nos. 

T=rand (n, m) ; 


for  aa=l:20:n  %For  rows  1  to  n,  by  incr.  of  20 

for  bb=l:20:m  %For  columns  1  to  m,  by  incr.  of  20 
if  (F (aa, bb) >f2 )  &  (T(aa,bb)>t) 

%copy (aa : aa+20 , bb : bb+20 ) =730 ;  %Set  square  area  of  copy  =  730 
beep 

beep  %pause(l) 

%p=p+l ; 

elseif  (F (aa, bb) >f 1)  &  (T(aa,bb)>t) 

copy (aa: aa+20, bb: bb+20 ) =0;  %Set  square  area  of  copy  =  0 
beep  %pause(l) 

%m=m+l ; 

end 

pause (0 . 00005) 

%imshow(copy) ; 

warp(copy);  %Redisplays  image  w/texture(?) 

title  ('Scanning  for  Mines') 
xlabel ( ' East ' )  ; 
ylabel (' North ') ; 

%legend (' plastic  mines'); 

%legend( 'metal  mines ' ) ; 

%imshov/(copy,  [  ]  )  ; 

end 

end 

title ('Mines  detected'); 


